Readmission prediction after colorectal cancer surgery: A derivation and validation study

Background Unplanned readmissions after colorectal cancer (CRC) surgery are common, expensive, and result from failure to progress in postoperative recovery. The context of their preventability and extent of predictability remains undefined. This study aimed to define the 30-day unplanned readmission (UR) rate after CRC surgery, identify risk factors, and develop a prediction model with external validation. Methods Consecutive patients who underwent CRC surgery between 2012 and 2017 at Christchurch Hospital were retrospectively identified. The primary outcome was UR within 30 days after index discharge. Statistically significant risk factors were identified and incorporated into a predictive model. The model was then externally evaluated on a prospectively recruited dataset from 2018 to 2019. Results Of the 701 patients identified, 15.1% were readmitted within 30 days of discharge. Stoma formation (OR 2.45, 95% CI 1.59–3.81), any postoperative complications (PoCs) (OR 2.27, 95% CI 1.48–3.52), high-grade PoCs (OR 2.52, 95% CI 1.18–5.11), and rectal cancer (OR 2.11, 95% CI 1.48–3.52) were statistically significant risk factors for UR. A clinical prediction model comprised of rectal cancer and high-grade PoCs predicted UR with an AUC of 0.64 and 0.62 on internal and external validation, respectively. Conclusions URs after CRC surgery are predictable and occur within 2 weeks of discharge. They are driven by PoCs, most of which are of low severity and develop after discharge. Atleast 16% of readmissions are preventable by management in an outpatient setting with appropriate surgical expertise. Targeted outpatient follow-up within two weeks of discharge is therefore the most effective transitional-care strategy for prevention.


Introduction
The readmission rate (RR) is a key performance indicator (KPI) that is widely used to evaluate the quality of hospital care delivered to a population [1][2][3]. Its well-validated association with the quality of healthcare, ease of calculation, and applicability to various patient cohorts underscores its clinical utility [1].
Unplanned readmissions (URs) have significant implications for both the patient and the healthcare system [4,5]. They result from a failure in the continued progression of patient recovery toward their preoperative baseline. This is supported by their association with poor patient outcomes such as increased morbidity, poor quality of life, and increased mortality [4,6]. URs are also associated with significant costs to the healthcare system, with an estimated economic impact of $17.4 billion per year in the US [7,8]. In the context of preventability, this is an important consideration for rationing the limited healthcare resources in the Covid-19 era.
Identification of robust risk factors for UR is critical for the development of readmission prediction models. The successful application of these models to high-risk individuals at discharge may facilitate targeted readmission-reduction interventions. Several studies have reported multiple risk factors for UR after colorectal surgery (CRS). However, this has not resulted in effective readmission prediction, primarily due to limitations arising from significant heterogeneity in this field of research [4,8,21,22]. The major sources included variations in sample size, sample sources, methodology, definitions, time intervals to readmission, and non-standardized categorization of both demographic and clinical risk factors.
This derivation and validation study aimed to define the 30-day UR rate after colorectal cancer (CRC) surgery; identify clinically relevant risk factors for UR; develop a clinically applicable prediction model; internally and externally validate this prediction model; and define readmission characteristics according to timing, diagnoses, and severity. To address previous limitations arising from heterogeneity, a homogenous cohort of consecutive patients undergoing curative colorectal cancer surgery in an elective setting was used as the study sample.

Patient selection
All patients who underwent elective CRC surgery between 2012 and 2017 at Christchurch Hospital were identified using the Binational Colorectal Cancer Audit (BCCA) database. The BCCA database was established by the Colorectal Surgical Society of Australia and New Zealand (CSSANZ). Consent to access de-identified data was obtained from the BCCA Operations Committee. Patients were excluded if they had recurrent colorectal cancer, stage 4 disease, noncurative surgery, or transanal excision of rectal tumours without a total mesorectal excision.

Clinical variables
The patients' medical records were retrospectively reviewed in parallel with Christchurch Hospital's clinical portal for additional data quality assurance. Missing data from the BCCA database were obtained to avoid incomplete case exclusion. The relevant clinical variables were extracted and tabulated. These were classified as preoperative, operative, and postoperative variables.
Preoperative variables included age, sex, and American Society of Anesthesiologists (ASA) score. Operative variables included surgical approach, surgery type, stoma formation, stoma type, and tumor site. Rectal tumors were defined as those with the lowest margin of <15 cm above the anal verge, as measured on sagittal magnetic resonance imaging. Postoperative variables included the occurrence of any postoperative complications (PoCs), length of hospital stay (LOS), discharge disposition, and stage (American Joint Committee on Cancer staging system) [23]. Postoperative complications were classified by number and severity based on the Clavien-Dindo (CD) classification grade [24]. Postoperative complications grade-3 and above were further categorized as "high-grade PoCs". The postoperative variable time-interval was defined as the day of surgery to discharge. All patients followed a standardised multimodal postoperative recovery pathway as per Enhanced Recovery after Surgery (ERAS).

Primary outcome
The primary outcome was defined as any unplanned readmission within 30 days of the index discharge. The decision for readmission was a collaboration between the Emergency Department (ED) physicians and the acute general surgery team based on the need for inpatient general surgery care in the ongoing work-up and management of presenting patients. Patients discharged directly from the Emergency Department were excluded. Mortality was excluded from the readmission analysis. The secondary outcomes were the timing, diagnosis, duration, and severity of unplanned readmissions.

Statistical analysis
Continuous variables are reported as medians with interquartile ranges (IQR) and categorical variables as whole numbers and percentages. Univariable analysis was performed to compare baseline characteristics between readmitted and non-readmitted patients. Appropriate statistical tests were selected based on variable type and parametricity. Multivariable logistic regression analysis was performed with stepwise backward selection, with a p-value greater than 0.05 used for exclusion of candidate variables. Regression coefficients for these model variables are reported as odds ratios (ORs) with 95% confidence intervals (CI). Multicollinearity between predictor variables was assessed by variance inflation factor (VIF), with scores of over 3 taken to indicate unacceptable collinearity. At least 15 events per candidate predictor were ensured to maintain statistical power for multivariable analysis, as recommended by the guidelines on prognostic factor research quality improvement [25]. A recent departmental audit demonstrated a readmission rate of 20%. A minimal sample size of 600 was hence required to power the multivariable analysis accordingly. Statistically significant predictor variables in the multivariable logistic regression analysis were incorporated into a predictive model. Statistical analysis was performed using R Studio software (version 3.5.2; www.rstudio.com).

Ethics
Ethical approval was obtained from the University of Otago Human Ethics Committee (#H17/ 148) following confirmation from the National Health and Disability Ethics Committee as a minimal risk observational study. Locality approval was obtained by the Canterbury District Health Board (ID 17255). All data was stored in a secure hospital server with access given to the authors of this study. All subsequent analysis was performed on de-identified datasets. Patient privacy and confidentiality was maintained on all phases of the study.

Model validation
The entire dataset was used to derive a predictive model and evaluate its performance. Internal validation was performed by bootstrapping to obtain a more precise estimate of model performance. The procedure involved repeating the modelling process with variable selection in 2000 bootstrap samples drawn with replacement from the original sample. This process quantified optimism secondary to overfitting and allowed for the adjustment of the regression coefficients and performance based on the estimated shrinkage factor. The model was then externally validated on a prospectively maintained sample set of patients based on the same inclusion criteria who underwent curative colorectal surgery between January 2018 and June 2019. The prospective validation design ensured consecutive patient selection and subsequent reduction of selection bias. Measures of model performance on this external sample set were then repeated and compared using similar statistical techniques.

Model performance
The performance of the predictive model in the derivation cohort was assessed through calibration and discrimination. Calibration was measured by the degree of agreement between the observed events, and the models predicted-risk probabilities using a calibration plot. An intercept of 0 and a slope of 1 on a calibration plot indicate perfect calibration. Discrimination refers to the ability of a model to differentiate between individuals who do and do not experience the primary outcome. This was measured using the area under the receiver operating characteristic (ROC) curve, where a value of 0.50 represented chance and 1 represented perfect discrimination.

Patient characteristics
A total of 701 patients underwent curative colorectal cancer surgery during the study period. The baseline characteristics, clinical variables, and frequencies are shown in Table 1. The median patient age was 72 years (IQR: 64-79 years). There were no significant differences in age, sex, ASA score, stage, or discharge disposition between the readmitted and non-readmitted patients. Anterior resection was the most common surgical procedure (44.1%) followed by colectomy (43.7%), APR (10.8%), and proctocolectomy (1.4%). The median LOS was 7 days (IQR: 5-10 days).

Unplanned readmission rate
Of the 701 patients discharged alive, 106 were readmitted within 30 days of the index discharge, giving an unplanned readmission rate of 15.1%. The timing of UR in relation to the index discharge is shown in Fig 1. The median time to UR after the index discharge was 8 days (IQR: 5-14 days). Approximately 75% of the URs (n = 80) occurred within two weeks of discharge, of which 45% occurred within one week after discharge (n = 48). The median duration of the URs was three days (IQR: 1-6 days).

Unplanned readmission characteristics
The readmission characteristics are shown in Table 2. The three most common readmission diagnoses were wound infection (27%), intra-abdominal collection (22%), and bowel obstruction or ileus (14%). The corresponding time to readmission and LOS for each diagnosis are listed in Table 2. All readmissions were for diagnoses related to the index surgery. When graded according to severity, 75% of unplanned readmissions were CD grade-1 and grade-2, as shown in Table 3.

Risk factors for unplanned readmission
Clinical variables and their corresponding univariable and multivariable odd's ratios are shown in Table 4. Length of Stay, surgical entry, surgery type, stoma, tumor site, and PoCs were found to be significant in the univariable analysis. The tumor site and stoma variables showed significant collinearity (VIF = 3.22). This collinearity was addressed by substituting the corresponding collinear variables in the analysis.

Prediction of unplanned readmission
To address collinearity, tumor site was chosen as a predictor variable instead of stoma. Two simple clinically applicable models for predicting unplanned readmission were assessed using the following predictor variables: tumor site, PoCs, and high-grade PoCs.
Model A consisted of two variables: tumor site and PoCs. Similarly, model B consisted of two variables: tumor site and high-grade PoCs.

Internal validation
Both models were internally validated on the derivation sample set using bootstrapping with replacement as previously described. A model consisting of all the significant variables in the univariable analysis was used for comparison. Comparative model performance was demonstrated using calibration plots and ROC curves. As shown in the calibration plot in Fig 2, both models overestimated the predicted probability of readmission, although model B was closer to the ideal slope of 1. Discrimination for both models on internal validation is shown by the ROC curves in Figs 3 and 4. The AUC for model A and model B were 0.66 and 0.64, respectively.
External validation. Both the models were externally validated on a prospective sample set of 156 patients. The overall unplanned readmission rate was 11.5% (n = 18). Baseline characteristics of the validation cohort are presented in the supporting information (S1 Table in S1 File). The discrimination ability of both models on external validation are shown on the ROC curves in Figs 3 and 4, in addition to their respective internal validation AUC. The AUC for Model A decreased from 0.66 to 0.58, compared with Model B , which decreased from 0.64 to 0.62. Therefore, Model B was demonstrated to be a superior predictive model.

Regression model B
The resulting logit regression model (Model B ) after internal validation with bootstrapping can be expressed as: where P readmission denotes the probability that a patient is readmitted within 30 days of index discharge. The tumor site variable had a value of "1" for rectal tumors and "0" for colonic tumors. Similarly, the CD3 + variable had a value of "1" for the occurrence of high-grade PoCs and "0" for no high-grade PoCs. Alternatively, the model can be represented according to the following equation to calculate the probability of readmission.

Discussion
The findings of this retrospective study demonstrated that 15% of the patients who electively underwent curative CRC surgery were readmitted within 30 days of index discharge, and all were for complications related to index surgery. The three most common readmission diagnoses were wound infection, intra-abdominal sepsis, and bowel obstruction. Approximately 75% of the URs occurred within 2 weeks of index discharge and were of low severity, as reflected by their CD grade and median LOS of 3 days. The three main risk factors for UR on multivariable analysis were stoma formation, rectal cancer, and the development of any PoCs before discharge.
Stoma formation was a significant risk factor for UR (adjusted OR 2.45, 95% CI 1.59-3.81). This is consistent with the findings of other systematic reviews of colorectal surgery readmissions [21,22]. In addition, the underlying correlation between stoma formation and readmission in these studies was attributed to stoma-related complications such as electrolyte abnormalities and renal failure. In contrast, these stoma-related complications accounted for only 5% of the readmissions in this study. Similarly, patients with rectal tumors were at an

PLOS ONE
increased risk of UR than those with colonic tumors (adjusted OR 2.11, 95% CI 1.48-3.52). This finding was supported by two studies that assessed this variable in relation to readmission and attributed it to the increased risk of morbidity associated with rectal dissection [13,26]. In line with this consideration, of the 271 rectal tumor resections in our study, >90 also had a stoma. The remaining stoma formation cases in our study (n = 28) were distal colon cancers that required rectal mobilization to obtain oncological margins, physiological intraoperative decompensation, positive anastomosis leak tests, and patient choice. This explains the high collinearity between rectal tumors and stoma formation in our analysis. Clinically relevant baseline characteristics of the rectal tumors from the overall cohort are shown in the supporting information (S2 Table in S1 File). Of 271 rectal tumor resections, > 90% required intraoperative stoma formation. Fifty-five percent of these cases developed a postoperative complication, and 25% resulted in unplanned readmission, in comparison to the overall unplanned readmission incidence of 15%. Based on these findings, rectal tumor and stoma variables represented cases associated with increased morbidity. This increased morbidity led to the development of PoCs, a proportion of which resulted in unplanned readmission.
In further support of this, the development of any PoC was a significant risk factor for UR (adjusted OR 2.27, 95% CI 1. 37-3.26). Studies outside the scope of this review across a range of surgical specialties have also reported an association between PoCs and UR [4,9,20,22,27]. In addition, a dose-response relationship between the number of PoCs and the risk of UR has been reported in studies not limited to colorectal surgery [27][28][29][30][31]. Further delineation of how the development of PoCs modulates the risk of UR was limited by the lack of reporting of PoC characteristics and heterogeneity from non-standardized definitions. To address this limitation, this study defined postoperative complications and their characteristics as shown in Tables 1, 2, and 5. A dose-response relationship was observed between the number of PoCs and the risk of UR, reaffirming the findings of previous studies in the colorectal cancer surgery setting. This trend lost statistical significance in patient with three or more PoCs, owing to the small number of these patients (<3.4%). When classified by severity, high-grade PoCs were a significant risk factor for UR.
The results of this study also defined the timing of PoCs in relation to the index readmission. All readmissions were for PoCs related to the index surgery. Of the 106 readmissions, 21 matched similar PoCs that occurred before index discharge ( Table 2). This demonstrated that 80% of readmissions were new postoperative complications that were either clinically silent or absent at the time of discharge and, therefore, developed after discharge in the community. This further supports the premise that URs after elective colorectal cancer surgery serve as surrogates for PoCs that develop after discharge.
Two prediction models were evaluated using statistically significant variables with the goal of being simple and clinically applicable. Both models used only two predictor variables, with model B demonstrating better discriminatory ability on external validation (AUC 0.62). Given the collinearity between the variables of tumor site and stoma, the former was chosen given the preoperative availability of this information. Two hundred eighty-three patients developed PoCs compared to 39 patients who developed high-grade PoCs. Therefore, the high-grade PoC variable provides a more precise target for patients at risk of UR than the PoCs variable. The percentage of readmissions deemed to be preventable varied significantly in the literature, ranging from 5% to 79% [1,14,27,[32][33][34]. This wide range is due to subjective estimation of preventability based on administrative datasets. The findings of this study show that the underlying aetiology of unplanned readmissions are postoperative complications, the majority of which develop after discharge. Postoperative complications remain a routine part of surgical care despite all preventative measures taken to minimize their occurrence. These measures include appropriate patient selection, optimization of preoperative comorbidities, fastidious surgical techniques, and appropriate postoperative protocols [35,36]. Despite patients following ERAS in this study, compliance was not measured due to limitations from the retrospective design. Compliance to ERAS has not been shown to influence readmission and is therefore unlikely to have influenced the primary endpoint of this study [37].
The clinical relevance of readmission prevention is blurred when considered solely in terms of their underlying aetiology. However, if a proportion of them can be successfully managed in the outpatient despite their underlying aetiology, they can then be deemed preventable, and clinically relevant. Wound infections that were diagnosed clinically without imaging comprised 16% of overall readmissions. This number reflects the minimum percentage of readmissions that could be preventable through outpatient management. This number is likely to be greater given that 41$ of all readmissions did not require diagnostic imaging, and 75% were of low severity (CD grade 1 and 2). Given the need for surgical expertise in their assessment and management, the interventional strategy of targeted outpatient follow-up within a two-weeks of index discharge is the transitional care strategy supported by this study for readmission prevention. The findings of this study support the idea that URs after CRC surgery are preventable, but the true extent of prevention through outpatient management is difficult to delineate due to its retrospective design and subjective nature of defining preventability. Further studies on readmission characteristics after implementing this transitional care strategy are required to further quantify this preventability.

Conclusions
The results of this study demonstrated that URs after colorectal cancer surgery are predictable. A proportion of these URs can be managed in the outpatient setting thus preventing and reducing readmissions. Patients with rectal cancer and those who developed high-grade PoCs during their index hospital stay had the highest risk of readmission and within two weeks of index discharge. Interventions to reduce readmissions, such as early clinical follow-up, may therefore be targeted toward these patients within this time interval. This strategy may prevent hospital readmissions though outpatient clinic management. The premise that readmissions represent a waste of healthcare resources is not supported by this study. While they represent a significant cost to the healthcare system, the need for surgical expertise in their management for the benefit of the patient justifies this expense.

Limitations
This study has several limitations. Its retrospective methodology makes it susceptible to bias from known and unknown confounders. The small sample size for some predictor variable categories may have allowed for the possibility of a type two error. The model development process was conducted according to the TRIPOD guidelines [38]. Differences were observed in the primary event rates of the validation dataset (11%, n = 18). Limited empirical evidence dictates a validation sample size of a minimum of 100 events and 100 non-events, with 250 events being preferable. Therefore, further validation studies are required to evaluate the formulated prediction models in this study.